Optimal route in a road traffic network

We develop a road traffic Cellular Automata model suitable to an urban environment. North, east, south and west car displacements are possible and road crossings are naturally implemented as rotary junctions. We consider the traffic in a small part of Amsterdam, a Manhattan city-like area. Besides the simulation, we apply a Reinforcement Learning technique to derive optimal routes. The Reinforcement learning approach implies to define a value function V on the state space X, and to learn the values for each state x ∈ X. To make a complete description of the system, the state space becomes too large. Therefore we propose the use of a Temporal-Difference Q-Learning algorithm, that avoids having to learn the complete transition model because the Q-value of a state can be related directly to those of its neighbours.